A Modular Approach to Knowledge Transfer Between Reinforcement Learning Agents

Reinforcement learning is a general approach to learning reactive control policies. It is an unsupervised learning technique, making it a candidate for use in system that adapts to changing tasks and environment by autonomously devising a new strategy. Unfortunately, reinforcement learning methods are slow to converge to a solution, rendering them impractical in most cases. The key shortcoming of most reinforcement learning systems is that they lack the ability to incorporate prior knowledge into the learning process. This is wasteful of any previously learned information that is related to the target task. This paper investigates a scheme for incorporating such prior knowledge into a reinforcement learning agent. The methodology, called Skill Advice Guided Exploration (SAGE), allows multiple, possibly conflicting, sources of knowledge to be incorporated simultaneously. Knowledge is abstracted as action selection advice within this framework. Accordingly, no assumptions must be made about the internal representations of the information, allowing knowledge sources to be as diverse as a human and a robot. Furthermore, a SAGE-based system can adapt according to new knowledge acquired during learning and is robust to incorrect information. The potential of this methodology is demonstrated on a set of discrete learning tasks. Results are presented showing that a SAGE-based system given correct information can outperform other approaches and that incorrect knowledge does not prevent the task from being learned. The benefits and limitations of this work are discussed as are possible extensions * Prepared in partial fulfillment of the requirements for the degree of Master of Science in Electrical and Computer Engineering.